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Regional Capacity Assessment 


Introduction and Purpose 


The primary purpose of the Regional Capacity Assessment (RCA) is to assist regional education agencies in their 
efforts to effectively support districts. The capacity of a regional education agency (REA) refers to the systems, 
activities, and resources that are necessary for a REA to be able to facilitate district-level implementation of 
Effective Innovations. 


The specific purposes of the RCA are to: 


1. Provide a State Education Agency (SEA) with information to monitor progress towards state, regional, 
and district capacity building goals 


2. Provide a Regional Education Agency (REA)! with a structured process for the development of a 
Regional Capacity Implementation Plan. 


3. Support the development of a Regional Implementation Team (RIT) for establishing district 
implementation capacity to achieve desired outcomes for students 


4. Support the development of a Regional Implementation Team (RIT) that can provide compensatory 
support functions to local districts (when the local district has limited resources or size) 


5. Provide state and regional leadership with a regular measure of the capacity for implementation and 
sustainment of Effective Innovations (Els) across the state 


Overview of Regional Education Agencies: 

A Regional Education Agency (REA) functions as a partner with local districts and the State Educational Agency 
(SEA). REAs provide training, guidance and support intended to enable school districts to operate efficiently 
and improve their student outcomes. Some examples of the types of services provided by a regional agency 
may include a combination of instructional supports (e.g., professional development in content and teaching 
practices, curriculum alignment), coordinating special education programming, instructional technology 
support services, or assistance with district operating procedures (e.g., payroll services, bus transportation 
services). REAs vary within and across states in the number of districts they serve and in types of services they 
provide to districts. 


Not all states have REAs. However, to provide the leverage needed to support the use of effective education 
methods, state education systems need to have a layer of support between the one SEA and the many 
districts. Regional agencies form Regional Implementation Teams (RITs) to develop internal capacity to 
support districts in selecting and using Effective Innovations in sustainable ways such that all students benefit 
from the innovations (http://sisep.fpg.unc.edu/resources/sisep-brief-5-leveraging-change-state-education- 


systems). Some smaller states do not require regional entities and can provide the leverage needed from the 
SEA (one SEA to a few districts instead of one SEA to many districts). In those states, the RIT functions can be 
developed in the context of SEA operating units. 


1 Each word or phrase that is in bold type is defined in the Glossary. 
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Schedule and Timeframe of RCA Administration 

The RCA is administered two times each year. After the initial baseline assessment, the RCA is re-administered 
at six-month intervals to refine the Regional Capacity Implementation Plan. Subsequent administrations of the 
RCA are used to monitor progress toward implementation capacity development. After each administration 
the team reviews previous RCA scores, reviews and updates data sources, and adjusts the Regional Capacity 
Implementation Plan as necessary. A regional agency may choose to administer the RCA more often during the 
year to help achieve targeted functions and implementation development purposes. 


Given the importance of the process and the complexity of the items, the estimated time to complete the RCA 
is one to two hours. Exact times will depend on the number of individuals participating, the familiarity of the 
team with the RCA items and the administration process, and the depth of understanding of current practices 
in districts in the region. The first use of the RCA typically takes more time than subsequent administrations. 
Preparing key documents prior to administration of the RCA reduces the time for administration (see the list of 
documents on the next page). 


Process and Key Roles 
The RCA administration process consists of: 


Introducing the RCA and its purpose 

Providing an overview of the process for completing the RCA 

Introducing the concepts or big ideas 

Reading each item aloud and providing any necessary clarification 

Facilitating the discussion and voting process 

Recording the score for each item 

Downloads data documents from sisep.org and sends them to the designated RIT team member no 
later than 5 business days post administration 

8. Summarizes notes and observations and sends them to the designated RIT team member no later than 
5 business days post administration. The notes are used to supplement the RCA scores to facilitate 
action planning by the State team 


NOW PWN EP 


The key roles required to carry out the RCA administration process are described in the table below: 
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A trained individual responsible for leading the discussion and adhering to the RCA 
Administration Protocol. The RCA Administrator is external to the regional education agency 
and does not vote. 


RCA 
Administrator 


An individual who has a relationship with the respondents and experience in the regional 
Facilitator education agency. The Facilitator supports the RCA Administrator by helping to contextualize 
items for respondents or provide examples of work in which the regional education agency has 
engaged. The Facilitator does not vote. 


An individual who records ideas shared for action planning and records any questions and 


Note Taker issues that are raised during administration. The Note Taker does not vote. 


Respondents include Regional Implementation Team (RIT) members and other staff 
Respondents | intentionally selected for their implementation knowledge, experience with the innovation 
being used, and leadership in the regional agency. Respondents vote. 


Observers are invited with permission of the RIT to learn about the RCA process and the 
current implementation capacity development activities in the region, or to assist the RCA 
Administrator and Facilitator. Observers do not participate in discussions during the 
administration of the RCA and do not vote. 


Observer 


Preparation for the RCA and Administration 


Prior to administering the RCA the following should be in place: 
1. The leadership of the regional education agency and the members of the Regional Implementation 
Team (RIT) agree to RCA administration and the commitment of time 


2. Materials to be assembled in preparation for RCA administration include: 
a. Previously completed RCA forms and data or reports from previous RCAs if applicable 
b. Blank copies of the RCA items (paper or electronic) accessible to all respondents 
c. Data sources to inform RCA assessment (Regional Capacity Implementation Plan needed at a 
minimum) 


Scoring 

During an in-person meeting the RCA Administrator uses the RCA Scoring Guide to encourage the respondents 
to discuss each item and come to consensus on the final score for each item. The respondents score each item 
on a 0-2 point scale using a simultaneous and public voting process. This type of voting process facilitates 
participation of all respondents and neutralizes any potential power influences in the voting process. When 
asked to vote (e.g., “ready, set, vote.”), respondents simultaneously hold up either two fingers to vote “fully in 
place,” one finger to vote “partially in place,” or a closed hand to vote “not in place.” 


A consensus score is recorded. Consensus means that voters in the minority can live with and support the 
majority decision on an item. If consensus cannot be reached after the first vote, the Facilitator encourages 
further discussion of the differences. A second vote is taken and the majority vote is recorded so that the 
results can be scored and graphed. If the team is unable to arrive at consensus on an item, the RIT may want 
to assure respondents have access to additional data sources prior to the next administration of the RCA. 


Research Basis and Outcomes from the RCA Completion 

The research basis of the RCA is derived from the implementation science research literature and the Active 
Implementation Frameworks. The Active Implementation Frameworks “help define what needs to be done 
(Effective Innovations), how to establish what needs to be done, who will do the work (effective 
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implementation), and establish the hospitable environment for the work (enabling contexts) to accomplish the 
positive outcomes” (Blase, Fixsen et al., 2005). The Active Implementation Frameworks are universal and apply 
to any attempt to use Effective Innovations. Once an Effective Innovation has been identified, and the 
implementation teams have been established, the work is guided by the use of Implementation Drivers, 
Improvement Cycles, and Implementation Stages. 


The Implementation Drivers assessed by the RCA are: 

e Leadership - Active involvement in facilitating and sustaining systems change to support the use of 
Effective Innovations through strategic communication, decisions, guidance, and resource allocation; 
promotes vision, visibility, priority, and policy. 

e Competency - Strategies to develop, improve, and sustain educators’ ability to use implementation science 
to support the use of an Effective Innovation as intended in order to achieve desired outcomes. 
Competency Drivers include: Staff Selection, Training, Coaching, and Fidelity (Performance Assessment). 

e Organization — Changing structures, roles, and processes to more precisely support the use of Effective 
Innovations and to collect, analyze, and use data in ways that result in continuous improvement of 
systems and supports for educators. Organization Drivers include: Decision Support Data System, 
Facilitative Administration, and Systems Intervention. 


The Improvement Cycles assessed by the RCA are based on Plan, Do, Study, Act Cycle (PDSA Cycle) logic that is 
used to develop and sustain excellence. PDSA Cycles can be used to establish solutions to problems that occur 
frequently (e.g., lack of scheduling support for coaches to meet as needed with teaching staff) or less often 
(e.g., insufficient leadership support for using an Effective Innovation in all schools in a district). Improvement 
Cycles provide a systematic approach to establishing plans and assessing results in complex environments such 
as schools, districts, regions, and states. 


The RCA assesses how Regional Implementation Teams (RITs) support districts that are in different stages of 
implementation. Implementation Stages outline the integrated, non-linear process of deciding to use an 
Effective Innovation and finally having it fully in place to realize the promised outcomes. Stages serve as a 
guide for the steps implementation teams need to take over time. Active implementation stages are 
Exploration, Installation, Initial Implementation and Full Implementation. 


RCA Items Mapping to Implementation Practices, Drivers, and corresponding subscales: 


a 
a 
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Organization 


e Decision Support Data Systems 


e  Facilitative Administration 6, 7, 11, 25, 26 
e Systems Intervention 27, 28 


e Stage-Based Functioning 10, 12,15 


Intended outcomes from RCA completion: 


1. Review and utilize the summary report with (a) Total score, (b) Subscale Scores, and (c) Item Scores to 
identify areas of strength and need 

2. Identify priorities to address within a plan 

3. Develop and create a Regional Capacity Implementation Plan that defines immediate and short term 
actions toward improving district capacity to implement Effective Innovations 


Administration Prerequisites 


SISEP provides training for each RCA Administrator. SISEP also arranges access to sisep.org, a web-based 
application that allows Regional Implementation Teams to complete, store, and view the results of the RCA. 
Team scores are entered electronically, and reports are generated to view (a) Total Scores, (b) Subscale Scores, 
and (c) Item Scores. These data are used to assess current implementation supports, monitor progress across 
time, and plan actions that will improve capacity to implement Effective Innovations. 


To access SISEP.org, please contact Caryn Ward at the email address below. 


SISEP.org User Types Description 

Agency Coordinator An Agency Coordinator can add and manage users for their organization, 
administer assessments, edit a submitted assessment, and view assessment 
reports. 

Assessment An Assessment Administrator can administer assessments, edit a submitted 

Administrator assessment, and view assessment reports. 

Read Only Read Only access only allows team members to view assessment reports. 


We ask that you let us know when you use the RCA. We want to support high fidelity use of the RCA, 
accumulate RCA data, and make use of your experience and data to improve and expand the assessment. 
Please respond to Caryn Ward. Thank you. 


Caryn Ward, Ph.D. 

Associate Director of Education and Measurement 

National Implementation Research Network 

Co-Director of State Implementation and Scaling up Center of Evidence Based Practices 
sisep@unc.edu, Reception: 919-843-5787 

FPG Child Development Institute 

University of North Carolina at Chapel Hill 
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Version 2.1 of the RCA 

The RCA development team uses an iterative change process based on feedback gathered from users (i.e., 
usability testing) and research findings to make continuous improvements to the RCA. When an instrument is 
in its early stages of development and use, the iterative change process occurs annually while the 
psychometric properties for the measure are being established. Changes to the measure are noted by 
changing its Version number. Minor refinements to the measure that are unlikely to impact user’s scores (i.e., 
misspellings, clarifications in introductory pages, slight re-wording to the scoring guide) result in a small 
incremental Version number change (i.e., 1.3 to 1.4) Significant changes to items and the corresponding 
scoring guide that impact meaning or the scaling structure of the instrument result in a large Version number 
change (i.e., 1.3 to 2.0). The changes that have been made to the RCA are significant; resulting in Version 2.0. 
More recent edits correcting minor typos brings this current RCA to Version 2.1. 


A summary of the changes made from Version 1.4 to 2.0 and their respective rationales are listed below: 


e Sequence change for item addressing the support of the formation of District Implementation Teams 
(former item # 12, current item #11), and the item addressing engaging in installation stage activities 
(former item #11, current item #12). For example, formation of implementation teams is an exploration 
stage activity; thus, this item now comes after the item assessing exploration stage activities and before 
the item assessing installation stage activities. 

e Addition of an item (new item #13) to the coaching subscale regarding the development and use of a 
coaching system. New item #13 outlines necessary infrastructure supports for the provision of effective 
coaching practices outlined in a coaching service delivery plan. 

e Changes in scoring criteria to enhance scoring specificity were made in items: 2, 4, 6, 8, 11, 17, 24, 27, 28 
(former item #s 12, 16, 23, 26, 27) 

e To provide increased clarity and further operationalization of the selection practices, significant changes 
were made in the scoring criteria on the item assessing selection practices of staff (current and former 
item #3). Also, edits were made to clarify that the selection practices assessed in the item encompasses all 
staff working within an REA. 

e Toenhance clarity and further operationalization of necessary stage-based activities, significant changes 
were made to scoring criteria for items assessing stage based activities of installation and initial 
implementation (current item #s 12, 15; former item #s 11, 14). 

e To provide clarification on the development and use of a process for selecting Els (#7), REA was specified 
as being the entity responsible for ensuring that process is used versus a RIT, who may have drafted the El 
selection process for REA leadership to approve for use. 

e To increase clarity and further operationalization of the necessary components of a coaching service 
delivery plan and in alignment with the new item assessing the use of a coaching system (new item #13), 
significant changes were made to the scoring criteria for the item assessing the use of coaching service 
delivery plan (current item #14, former item #13) 

e Significant changes were made to the scoring criteria for the item assessing communication practices 
(current item #25, former item #24) to include “use” of a communication plan in addition to inclusion of 
the plan’s necessary components. 

e Additional examples of potential data sources were added to items: 6, 7, 8 and 16 (former item # 15) 
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e Toenhance clarity, minor changes in wording of score descriptors of several items were made. Items 
include: 5, 7, 10, 17, 19, 20, 21, 22, 23, 24, 26, and 28 (former item #s 5, 7, 10, 16, 18, 19, 20, 21, 22, 23, 
25, 27). 
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RCA Administration Fidelity Checklist 


Protocol Steps: To be completed by the RCA Administrator to help assure Step 
consistency in the use of the RCA. This information is entered on SISEP.org. Completed? 
Y=Yes; N=No 


N/A= unsure or 
not applicable 


1. Respondents Invited- RCA Administrator and/or Facilitator invites RIT members and other Y N N/A 
knowledgeable participants to rate the items (REA leaders, and others) 
2. Materials Prepared in Advance- RCA Administrator and/or Facilitator ensures that copies Y N N/A 


(paper or electronic) of a blank RCA are available for each member and ensures that a room is 
set up with a device to connect to the internet, a projector, an internet connection, and a 
conference phone (video if possible) for any participants joining remotely 


3. Overview- RCA Administrator provides a review of RCA, purpose, and instructions for voting Y N N/A 
4. Administration- Blank RCA is available for entire team to view. Y N N/A 
5. Administration- Each question is read aloud. After the RCA Administrator reads a question, Y N N/A 
the Facilitator says, “ready, set, vote” and all respondents vote simultaneously and publicly 
(e.g., hold up 2 fingers to vote “fully in place,” 1 finger to vote “partially in place,” or a closed 
hand to vote “not in place or not yet” or holds up a card with the number 0, 1, or 2) 
6. Administration- RCA Facilitator tallies the votes and notes agreement or discrepancies foreach Y N N/A 


question. The Note Taker records the essence of the issues and discussions. 
7. Consensus- If complete agreement is reached, move on to the next question. If not, the RCA Y N N/A 
Facilitator invites an open, brief discussion of the reasons for differences in scoring. The group 
is asked to vote again. The vote can occur multiple times at the discretion of the Facilitator. 
The goal is to reach consensus. Consensus means that the minority voters can live with and 
support the majority decision on an item. If the minority persists in not being able to live with 
the majority vote, the majority vote is recorded (further discussion should be scheduled for a 
time after the RCA is completed). The Note Taker records the issues and any decisions for 
further meetings and discussions. 


8. Recording- RCA Administrator/Facilitator publicly documents each scoring decision Y N N/A 

9. Data summary- After the last question has been asked and answered, RCA Y N N/A 
Administrator/Facilitator/or Note Taker generates reports to display graphs and scores 

10. Review- While viewing the graphs, RCA Administrator/Facilitator prompts the team ina Y N N/A 


discussion of the results to identify strengths and opportunities. If a repeated administration, 
RCA Administrator highlights all of the subscales that moved in a positive direction and 
celebrates progress toward 80% or better subscale scores 

11. Region Status Review- RCA Administrator/Facilitator initiates a discussion of updates on Y N N/A 
achievements, progress, and major milestones or barriers that have occurred since previous 
administration 

12. Action- RCA Administrator/Facilitator asks respondents to discuss three domains they would Y N N/A 
like to set as action planning agenda items for their regular meetings 

13. Planning- If there is not sufficient time for #11 and #12, the RCA Administrator/Facilitator Y N N/A 
ensures that a date and time are scheduled for follow-up 

14. Conclusion- RCA Administrator/Facilitator thanks the team for their openness and for sharing Y N N/A 
in the discussion 


Comments/Notes: 
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Scoring Form 


Regional Education Agency: Date: 

RCA Administrator: Facilitator: 

RIT Members and other Respondents: 

Directions: The Regional Implementation Team (RIT) and other Respondents complete the Regional 

Capacity Assessment (RCA) together by using the RCA Scoring Guide to discuss each item and come to 

consensus on the final score for each item. If the team is unable to arrive at consensus, additional data 

sources for each item are documented in the RCA Scoring Guide and should be used to help achieve 

consensus. Scores are recorded on this Scoring Form below and within the Excel Data Capture Form. 

Score 
Item Fullyin | Partially in Not 
Place (2) Place (1) Yet (0) 

1. There is a Regional Implementation Team (RIT) to support districts in 2 1 0 
developing implementation capacity 

2. RlITincludes an individual with executive leadership authority 2 1 

3. The Regional Education Agency (REA) uses a process for selecting staff 2 1 
(internal and/or external) who support implementation capacity efforts 

4. Regional staff who support implementation have a plan to continuously 2 1 0 
strengthen skills 

5. RlTincludes a designated coordinator(s) 2 1 0 

6. RIT uses an effective team meeting process 2 1 0 

7. REA has a written process for selecting Effective Innovations (Els) 2 1 0 

8. RIT utilizes a process to operationalize Els 2 1 0 

9. REA allocates resources to support the development of district 2 1 0 
implementation capacity 

10. RIT engages in Exploration Stage activities with districts to develop 2 1 0 
implementation capacity 

11. RIT provides support for the composition of District Implementation 2 1 0 
Teams (DITs) 

12. RIT engages in Installation Stage activities with districts to develop 2 1 0 
implementation capacity 

13. REA has a coaching system to support districts in developing 2 1 0 
implementation capacity 

14. RIT uses a coaching service delivery plan to support district 2 1 0 
implementation teams 

15. RIT engages in Initial Implementation Stage activities with districts to 2 1 0 
develop implementation capacity 

16. RIT secures training for district/building staff 2 1 0 
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17. RIT supports districts in the use of fidelity measures 2 1 0 

18. RIT has access to relevant data 2 1 0 

19. RIT has a process for using data for decision making 2 1 0 

20. REA has a regional implementation plan for developing regional 2 1 0 
implementation capacity 

21. RIT has an implementation plan for developing district implementation 2 1 0 
capacity 

22. RIT continuously improves the use of implementation plans 2 1 0 

23. RIT uses training effectiveness data 2 1 0 

24. RIT uses coaching effectiveness data 2 1 0 

25. RIT uses a communication plan 2 1 0 

26. RIT uses a process for addressing internal barriers 2 1 0 

27. RIT uses a process to report policy relevant information to the State 2 1 0 
Education Agency (SEA) 

28. RIT provides stakeholders a status report on capacity development 2 1 0 
efforts 
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Scoring Guide 


RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
1. Thereisa A team is developed and is A team is developed and is There is not a Regional List of team 
Regional representative of cross- representative of cross- Implementation team (RIT) members, roles, 
Implementation | departmental perspectives (e.g., | departmental perspectives (e.g., and job titles 
Team (RIT) to general education and special general education and special RIT appears on 
support districts | education) education) organization 
in developing -AND- chart 
implementation Team members are selected for RIT position 
capacity their experiences in: descriptions 
e Attempted use of effective 
innovations 
e Positive working 
relationships with district 
leadership and staff 
e Sufficient dedicated time to 
perform RIT functions 
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RCA Item 


Fully in Place 
2 points 


Partially in Place 
1 point 


Not in Place or Not Yet 
0 points 


Potential Data 
Source 


RIT includes an 
individual with 
executive 
leadership 
authority 


RIT membership includes at least 
one executive leader who can 
make significant decisions within 
the Regional Education Agency 
(REA) to support districts in their 
implementation capacity efforts 
(e.g., time, internal structures, 
funds, personnel) without 
consulting with a higher authority 


-AND- 
Attendance at meetings is regular 
-AND- 


When scheduling conflicts 
occasionally occur, the leader 
makes sure (s)he is provided with 
relevant information (e.g., 
decisions needed and potential 
barriers that need to be 
addressed by other REA leaders) 
within 1-2 days after the meeting 


RIT membership includes at least 
one executive leader who can 
make significant decisions within 
the Regional Education Agency 
(REA) to support districts in their 
implementation capacity efforts 
(e.g., time, internal structures, 
funds, personnel) without 
consulting with a higher authority 


-AND- 
Attendance at meetings is regular 
-AND- 


When scheduling conflicts 
occasionally occur, there is not a 
mechanism for the leader to be 
provided with relevant 
information within 1-2 days after 
the meeting 


There is not an executive leader 
represented on the RIT 


-OR- 


The executive leader’s lack of 
attendance at RIT meetings 
impedes the team’s ability to 
complete the activities needed to 
support districts 


Executive leader 
job description 


List of team 
members, roles, 
and job titles 


Communication 
plan linking REA 
executive 
leadership to 
other 
departments and 
structures 
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RCA Item 


Fully in Place 
2 points 


Partially in Place 
1 point 


Not in Place or Not Yet 
0 points 


Potential Data 
Source 


The Regional 
Education Agency 
(REA) uses a 
process for 
selecting staff 
(internal and/or 
external) who 
support 
implementation 
capacity efforts 


Job descriptions exist that 
describe pre-requisite knowledge 
and responsibilities required 


-AND- 


The selection process includes 

the following: 

e Performance assessment 
asking individual(s) to 
demonstrate the required 
knowledge, skills, and 
abilities (e.g., simulated 
activity, role-plays, product 
development) 

e Feedback provided to 
individuals for the purpose of 
assessing the individuals’ use 
of the feedback (e.g., 
incorporating feedback into 
subsequent portions of the 
selection process) 


-AND- 
Interview protocol is revised as 
needed to improve the selection 
process 


Broad job descriptions exist 


-AND- 


The selection process is aligned 
to pre-requisite knowledge and 
responsibilities but does not 
meet the criteria outlined in the 
2-point response 


Broad job descriptions exist 


-AND- 


Generic job interview protocol 
used during the selection process 
does not assess pre-requisite 
knowledge and responsibilities 
required (e.g., similar protocol 
for any position) 


Job descriptions 


Interview 
protocol 
(including 
procedures used 
during the 
selection 
process) 
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RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
4. Regional staff who | All staff (newly selected and All staff have a written None of the staff have a written _| Staff professional 
support existing regional staff) assigned professional learning plan that professional learning plan learning plans 
implementation to develop regional and district includes at least 2 of the criteria -OR- 
have a plan to implementation capacity have a | outlined in the 2-point response | written professional learning 
continuously written professional learning plans do not meet the criteria 
strengthen skills plan that includes: -OR- sutiwediin the 2pombressensé 
e Areas for further Some staff have a professional 
development learning plan that includes all the 
e Type of professional learning | criteria outlined in the 2-point 
that will be provided to response 
develop knowledge, skills 
and abilities (e.g., attending 
a professional learning 
session, independent reading 
with structured dialogue to 
reflect on readings) 
e Timeline for completing the 
learning 
e Person(s) assigned to 
support staff 
-AND- 
Time is allocated for staff to 
develop knowledge, skills, and 
abilities that are outlined in the 
plan 
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RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
5. RiTincludesa RIT includes a designated RIT includes a designated RIT does not include a Coordinator job 
designated coordinator(s) who performs a coordinator(s) who performs a designated coordinator description 


coordinator(s) 


lead role on the RIT by: 

e Preparing for and facilitating 
the RIT meetings 

e Developing meeting agenda 

e Monitoring completion of 
assigned actions and 
implementation plan 


- AND- 


The coordinator(s) obtain direct 
contact to executive leader(s) on 
a regular basis (e.g., before and 
after RIT meetings and meetings 
with district leaders and District 
Implementation Team, DITs) 


-AND- 


Coordinator(s) uses knowledge 
about implementation research 
in order to make 
recommendations to the RIT and 
the executive leader overseeing 
the RIT 


-AND- 
Individual(s) providing 


coordination have adequate time 
to fulfill responsibilities 


lead role on the RIT by: 

e Preparing for and facilitating 
the RIT meetings 

e Developing meeting agenda 

e Monitoring completion of 
assigned actions and 
implementation plan 


-AND- 
The coordinator(s) obtain direct 
contact to executive leader(s) on 
a regular basis (e.g., before and 
after RIT meetings and meetings 
with district leaders and District 
Implementation Team, DITs) 


-OR- 


The coordinator(s) does not 
perform a lead role on the RIT 


-OR- 
The coordinator(s) does not 
obtain direct contact to 
executive leader(s) on a regular 
basis (e.g., before and after RIT 
meetings and meetings with 
district leaders and DITs) 


Identification of 
who the 
executive 
leader(s) are that 
work with the 
coordinator 


Time allocated 
for the 
coordinator role 
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used (e.g., facilitator, 
recorder, time keeper, norms 
monitor) 

e Process is in place for absent 
staff to receive updates 
within 48 hours following the 
meeting 

e Completes assignments and 
documents progress outlined 
on an action plan within 
designated timelines 


used (e.g., facilitator, 
recorder, time keeper, norms 
monitor) 


RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
6. RIT uses an RIT uses the following effective RIT uses the following effective RIT meeting processes do not Meeting 
effective team meeting processes: meeting processes: meet the criteria in the 2 or 1 schedule 
meeting process e Meets in person at least e Meets in person at least point responses. 
monthly or more frequently monthly or more frequently Meeting 
depending on amount of depending on amount of Agendas, 
work work Minutes, and 
e Meeting roles and e Meeting roles and Attendance 
responsibilities are responsibilities are 
consistently assigned and consistently assigned and Action Plan 


Documentation 
of the process for 
updating absent 
team members 
(i.e., Terms of 
Reference) 
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variables: 

e Need for the El 

e = Fit and alignment with other 
Els 

e Resources needed to fully 
implement 

e Capacity within the regional 
agency to successfully 
implement the El 

e Evidence to demonstrate 
effectiveness 

e _El’s readiness for replication 


-AND- 


Process is consistently used 


variables: 

e Need for the El 

e = Fit and alignment with other 
Els/initiatives/programs 

e Resources needed to fully 
implement 

e Capacity within the regional 
agency to successfully 
implement the El 


-OR- 


The process is not consistently 
used 


RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
7. REAhasa written | Written process is in place Written process is in place Written process is not in place Guidance 

process for documents 
selecting Effective -AND- -AND- outlining the 
Innovations (Els) process 

The process to select Els includes | The process to select Els includes 

an analysis of the following an analysis of the following Written 


documentation 
showing how the 
selection process 
has been used 
within the past 2 
years 
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RCA Item 


Fully in Place 
2 points 


Partially in Place 
1 point 


Not in Place or Not Yet 
0 points 


Potential Data 
Source 


8. RIT uses a process 


to operationalize 
Els 


The process to operationalize Els 
(e.g., Practice Profiles) includes: 
e Clear description of the El 
that includes its underlying 
philosophy and guiding 
principles 
e Clear description of the 
target population 
e = Identification of the critical 
components 
e Operational definition of 
each of the critical 
components for: 
o Expected use 
o Developmental 
variations 
o Unacceptable 
variations 


-AND- 


The process is consistently used 
either for Els that are not already 
operationalized or Els whose 
components are inadequately 
operationalized 


The process to operationalize Els 

includes: 

e Identification of the critical 
components 

e Operational definition of 
each of the critical 
components for: 

o Expected use 


-OR- 


The process is used 
inconsistently 


There is not a process to 
operationalize Els 


-OR- 
The process does not meet the 


criteria in the 2 or 1-point 
responses 


Documentation 
of the process 
(Guidance 
documents) 


Documentation 
of at least one 
operationalized 
El (e.g., Practice 
Profile; 
Innovation 
Configuration) 


9. REAallocates There is evidence of There is evidence of effort to There is no evidence of General fund 
resources to commitment to support the secure the commitment to commitment or effort to secure budget 
support the regional agency’s focus on support the regional agency’s commitment to support the allocations 
development of developing district focus on developing district regional entity developing Grant budget 
district implementation capacity through | implementation capacity district implementation capacity | allocations 
implementation the allocation and / or re- 
Capacity prioritization of resources (time, 

personnel, materials, etc.) 
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RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
10. RIT engages in Exploration Stage activities with | Exploration Stage activities with RIT does not engage in Documentation 
Exploration Stage | districts include: districts include: Exploration Stage activities with | outlining 
activities with e Meeting(s) with district e Meeting(s) with district districts Exploration Stage 
districts to leadership (executive leaders leadership (executive leaders activities with 
develop and administrators) to and administrators) to -OR- districts as 
implementation discuss the benefits of discuss the benefits of described in the 
capacity developing district developing district The Exploration Stage activities 2-point response 
implementation capacity implementation capacity with districts only include a 
e Decision-making protocol to |e Collection of baseline district | meeting with district leadership 
arrive at a mutual decision capacity data (e.g., DCA) executive leaders and 
whether to proceed with e Action planning based on administrators) to discuss the 
district implementation baseline capacity data benefits of developing district 
capacity work implementation capacity 
e Collection of baseline district 
capacity data (e.g., DCA) -OR- 
e Action planning based on 
baseline capacity data that Districts are assigned without the 
includes next steps to benefit of engaging in 
engage in installation stage Exploration Stage-based 
activities activities to arrive at a mutual 
e Readiness activities that will decision to proceed 
get districts prepared to 
engage in Installation Stage 
work 
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RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
11. RIT provides RIT members provide support in | RIT members provide guidance RIT members do not provide List of DIT 


support for the 
composition of 
District 
Implementation 
Teams (DITs) 


23 


the formation of DITs by focusing 

on the following: 

e Cross-departmental team 
composition (e.g., general 
education and special 
education) 

e Team size 

e Selection of the personnel to 
perform key roles on the 
team (executive leadership 
support and coordination) 

e Experience of team members 
(e.g., successful use of Els, 
positive relationships with 
staff, adequate time to fulfill 
responsibilities) 


-AND- 


Selection results for DIT 
membership are used to 
strengthen the Installation Stage 
activities 


and ongoing feedback on the DIT 

composition by focusing on the 

following: 

e Cross-departmental team 
composition (e.g., general 
education and special 
education) 

e Team size 


guidance and ongoing feedback 
on the DIT composition in a way 
that meets the conditions of the 
1- or 2-point responses 


members and job 
titles 


Team selection 
criteria 


Team selection 
protocols 
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activities with 
districts to 
develop 
implementation 
Capacity 


e District Implementation 
Team (DITs) development 
(e.g., effective and efficient 
meeting processes) 

e El fluency 

e Communication protocols to 
groups / teams 

e Barrier removal process 

e Coaching system to select 
and support coaches to 
coach the El concepts / skills 

e Process to align Els 

e Process to review, select, 
and de-select Els 

e Development of an 
Implementation Plan 

- AND - 

e Ongoing professional 
learning and coaching for the 
DIT and its members 

-AND- 

e Co-facilitation of DIT 
meetings to assist the team 
in applying their learning to 
support schools in the 
effective use of El 
components 


criteria outlined in the 2-point 
response 


- AND- 
Ongoing professional learning 


and coaching for the DIT and its 
members 


meet the conditions of the 2- or 
1-point response 


RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
12. RIT engages in Installation Stage activities with | Installation Stage activities RIT does not engage in Documentation 
Installation Stage | districts include the following: include at least four of the Installation Stage activities that outlining 


Installation Stage 
based 
implementation 
activities and 
which districts 
have received 
those supports 
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e Coaching definition 

e Conditions that warrant 
coaching 

e Coach pre-requisite 
knowledge and 
responsibilities 

e Coach selection guidelines 


(for existing or new staff that 


will be assigned districts to 
coach) 

e Allocation of time for REA 
staff to provide coaching to 
DITs 

e Statement clarifying the 
coaches’ decision-making 
authority 

e Coaching service delivery 
plan 

e Communication protocol 
between coaches and other 
groups / teams internal and 
external to the REA 

e Supervision and 
accountability structures 


-AND- 
The coaching system is 
consistently used 


-OR- 
The coaching system includes at 
least 6 of the criteria outlined in 
the 2-point response 


-OR- 


The coaching system is 
inconsistently used 


-OR- 


The coaching system is not used 


RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
13. REAhasa The coaching system is The coaching system is in the There is not a coaching system Coaching system 
coaching system developed to ensure districts process of being developed to document 
to support have equitable access to high- ensure districts have equitable -OR- 
districts in quality coaching to develop access to high-quality coaching 
developing implementation capacity to develop implementation The coaching system does not 
implementation -AND- capacity meet the criteria of the 2- or 1- 
capacity The coaching system includes: point response 
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coaching service 
delivery plan to 
support district 
implementation 
teams 


DITs are documented and 

includes: 

e Stage-based coaching 
concepts that distinguish 
supports to districts 

e Continuum of coaching 
strategies necessary to 
develop the team’s 
knowledge, skills, and 
abilities (e.g., observation, 
prompting, modeling, 
assistance in adapting to 
local context) 

e Coaching effectiveness 
measures (district capacity 
data, observation, products, 
coaching satisfaction survey 
data) 

e Frequency of coaching 

e Expectations for the coach’s 
preparation 

e Guidelines for providing 
feedback (e.g., conceptual 
feedback components, 
specific behaviors, timelines 
for providing written 
feedback) 

-AND- 

Adherence to the coaching 


service delivery plans is reviewed 


three times a year 


DITs are documented and 

includes: 

e Stage-based coaching 
concepts that distinguish 
supports to districts 

e Continuum of coaching 
strategies necessary to 
develop the team’s 
knowledge, skills, and 
abilities (e.g., observation, 
prompting, modeling, 
assistance in adapting to 
local context) 

e Coaching effectiveness 
measures (district capacity 
data, products, coaching 
satisfaction survey data) 


does not exist 

-OR- 
Coaching supports provided to 
DITs do not meet the criteria in 
the 1-point response 


-OR- 


The coaching service delivery 
plan is not being used 


RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
14. RIT uses a Coaching supports provided to Coaching supports provided to Coaching service delivery plan Sample of 


coaching service 
delivery plans 
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RCA Item 


Fully in Place 
2 points 


Partially in Place 
1 point 


Not in Place or Not Yet 
0 points 


Potential Data 
Source 


15. RIT engages in 


Initial 
Implementation 
Stage activities 
with districts to 
develop 
implementation 
capacity 


Initial Implementation Stage 
activities with districts include 
on-going professional learning 
and coaching in the following: 

e Assistance with exploration 
activities with buildings 

e Supports DIT in forming 
Building Implementation 
Teams (BITs) (explanation of 
roles and responsibilities for 
supporting staff’s use of Els; 
committee / team inventory) 

e Data and data system 
support (installation of 
measures and data systems, 
initial data collection) 

e Data analysis and use for 
both the DIT and BITs to 
determine the effectiveness 
of efforts 

e On-going refinements of 
district processes and 
procedures for El use 


-AND- 


RIT representation meets with 
the DITs at least monthly during 
the academic school year 


Initial Implementation Stage 
activities with districts include 
on-going professional learning 
and coaching in the following: 

e Supports DIT in forming 
Building Implementation 
Teams (BITs) (explanation of 
roles and responsibilities for 
supporting staff’s use of Els; 
committee / team inventory) 

e Data analysis and use for 
both the DIT and BITs to 
determine the effectiveness 
of efforts 


RIT does not engage in Initial 
Implementation Stage activities 
that meet the conditions of the 
2- or 1-point response 


Documentation 
outlining Initial 

Implementation 
Stage activities 

with districts 
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28 


Highly-competent individuals 
provide trainings (e.g., trainers 
who have deep knowledge in the 
areas they are providing training, 
effective presentation delivery 
skills) 

-AND- 


Trainings are skill-based, include 
opportunities for 
practice/behavioral rehearsals 
when applicable, and provide 
feedback to participants 


Highly-competent individuals 
provide trainings (e.g., trainers 
who have deep knowledge in the 
areas they are providing training, 
effective presentation delivery 
skills) 


RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
16. RIT secures Training is prioritized for district | Training is prioritized for district | Training is prioritized for district Professional 
training for district | staff in: Staff in: staff in: learning: 
staff e Implementation science e Implementation science e El specific data, systems, and | schedule, 
practices practices practices outlines, 
e El specific data, systems, and | e El specific data, systems, and agendas, 
practices practices participant 
materials 
-AND- -AND- 


Professional 
learning 
participant 
evaluations 


Presenter 
qualifications 
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RCA Item 


Fully in Place 
2 points 


Partially in Place 
1 point 


Not in Place or Not Yet 
0 points 


Potential Data 
Source 


17. RIT supports 


districts in the use 
of fidelity 
measures 


RIT supports districts in selecting 
research-validated fidelity 
measures that are highly 
correlated with (i.e., predictive 
of) intended outcomes for Els if 
available 

-OR- 


In the absence of available 
research-validated fidelity 
measures, the RIT has 
operationalized the El 


-AND- 


RIT supports districts to develop 
their capacity to use fidelity 
measures as recommended (e.g., 
proactive orientation of staff to 
the fidelity measure, 
administration, scoring, data 
entry, report generation, and 
analysis) 


RIT supports districts in selecting 
research-validated fidelity 
measures that are highly 
correlated with (i.e., predictive 
of) intended outcomes for Els if 
available 


-OR- 


In the absence of available 
research-validated fidelity 
measures, the RIT has 
operationalized the El (e.g., 
developed a practice profile) 


RIT does not support districts in 
selecting or using fidelity 
measures for Els 


-AND- 


In the absence of the availability 
of research validated fidelity 
measures, the RIT does not have 
a plan to operationalize the El 


Fidelity measure 


Practice profile 
or innovation 
configuration 


Data (e.g., local 
or published) 
demonstrating 
that fidelity 
predicts intended 
outcomes 
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analyze: 

e Regional capacity data (e.g., 
RCA) 

e District capacity data (e.g., 
DCA, Assessing Drivers Best 
Practices, DIT 
Implementation Plan data) 

e Scale-up data (e.g., Stages of 
Implementation Analysis: 
Where are We Now) 

e El fidelity data 

e Student outcome data (e.g., 
universal screening data, 
progress monitoring data, 
and summative assessment 
data) 


data: 

e Regional capacity data 
e District capacity data 
e Scale-up data 

e El fidelity data 

e Student outcome data 


in the 2-point response 


RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
18. RIT has access to All of the following relevant data | The RIT has access to at least The RIT has access to less than Sample data 
relevant data are accessible for the RIT to three of the following types of three of the data sources listed reports 
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RCA Item 


Fully in Place 
2 points 


Partially in Place 
1 point 


Not in Place or Not Yet 
0 points 


Potential Data 
Source 


19. RIT has a process 


for using data for 
decision making 


RIT uses a problem-solving 
process (e.g., Improvement 
Cycles) at least three times a 
year 


-AND- 


The process for using data 

includes: 

e =Analysis of all new data, 
resulting in a summary of 
celebrations and precise 
problem statements 


RIT uses a problem-solving 
process (e.g., Improvement 
Cycles) at least once a year 


-AND- 


The process for using data 

includes: 

e Analysis of all new data, 
resulting in a summary of 
celebrations and precise 
problem statements 


RIT does not have a problem- 
solving process 


-OR- 


RIT uses a problem-solving 
process (e.g., Improvement 
Cycles) that does not meet the 
conditions of the 2 or 1-point 
response 


Evidence of the 
problem-solving 
process 


Evidence that 
improvement 
cycles resulted in 
refinement of the 
implementation 
plan 


e Generation of hypotheses as 
to the factors contributing to 
the problem 

e Analysis of data to validate 
hypotheses or generate new 


hypotheses 

e Refinement of the 
implementation plan 
(S.M.A.R.T. goals and 
activities) to address the 
problem 
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RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
20. REA has an The plan’s primary purpose isto | The plan’s primary purpose isto | There is not a plan REA 


implementation 
plan for 
developing 
regional 
implementation 
capacity 


develop REA implementation 
capacity 


-AND- 


The plan has been approved by 
REA executive leadership 


-AND- 
Regional capacity data (e.g., RCA) 
are used to develop and update 
the plan 

-AND- 
The plan’s goals are S.M.A.R.T. 


-AND- 


The plan includes specific 
activities to achieve the goals 


develop REA implementation 
Capacity 


-AND- 


The plan has been approved by 
REA executive leadership 


-AND- 
Regional capacity data (RCA) are 


used to develop and update the 
plan 


-OR- 


The plan has not been approved 
by REA executive leadership 


implementation 
plan 


Record of plan’s 
approval 
(meeting minutes 
or other written 
communication, 
signature) 
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implementation 
plan for 
developing district 
implementation 
capacity 


assist districts to develop 
implementation capacity that 
can be generalized across Els 


-AND- 


The plan has been approved by 
REA executive leadership 


-AND- 


Data are used to develop the 

plan: 

e District capacity data (e.g., 
DCA data) 

e Scale-up data (e.g., each 
district and school’s stage of 
implementation) 

e Fidelity data 

e Student outcome data 


-AND- 
The plan’s goals are S.M.A.R.T. 
-AND- 


The plan includes specific 
activities to achieve the goals 


assist districts to develop 
implementation capacity that 
can be generalized across Els 


-AND- 


The plan has been approved by 
REA executive leadership 


-AND- 


At least two data sources are 
used to develop and update the 
plan 

e District capacity data (e.g., 
DCA data) 

e Scale-up data (e.g., each 
district and school’s stage of 
implementation) 

e = Fidelity data 

e Student outcome data 


-OR- 


The plan has not been approved 
by REA executive leadership 


RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
21. RIT has an The plan’s primary purpose isto | The plan’s primary purpose isto | There is not a plan RIT 


implementation 
plan 


Record of plan’s 
approval 
(meeting minutes 
or other written 
communication, 
signature) 
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RCA Item 


Fully in Place 
2 points 


Partially in Place 
1 point 


Not in Place or Not Yet 
0 points 


Potential Data 
Source 


22. RIT continuously 


improves the use 
of implementation 
plans 


Use of the implementation plans 
is reviewed a minimum of three 
times per year 


-AND- 


The review includes 

documentation of: 

e Status of activities (e.g., 
completed, partial, not- 
started) 

e Reasons activities are not 
completed (e.g., insufficient 
funding, training) 

e Team decisions (e.g., provide 
required resources to 
complete activities, 
communicating barriers, next 
steps) 


-AND- 


Plans are modified when data 
suggest the need 


Use of the implementation plans 
is reviewed at least two times 
per year 


-AND- 


The review includes 

documentation of: 

e Status of activities (e.g., 
completed, partial, not- 
started) 

e Reasons activities are not 
completed (e.g., insufficient 
funding, training) 

e Team decisions (e.g., provide 
required resources to 
complete activities, 
communicating barriers, next 
steps) 


Use of the implementation plans 
are reviewed once a year or are 
not reviewed 


Documentation 
of plans use and 
improvement 
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RCA Item 


Fully in Place 
2 points 


Partially in Place 
1 point 


Not in Place or Not Yet 
0 points 


Potential Data 
Source 


23. RIT uses training 


effectiveness data 


RIT uses the following types of 

training effectiveness data: 

e Pre and post assessment of 
knowledge and skills 

e Observation of participants 
practicing and applying newly 
acquired skills 

e Trainer performance 
feedback (e.g., feedback on 
training content, training 
process, delivery) 


-AND- 


Training effectiveness data 

are analyzed to: 

e Inform improvements to the 
content and delivery 

e = Inform improvements in 
recruitment and selection, 
training, coaching, and other 
implementation supports 


RIT uses two of the following 

types of training effectiveness 

data: 

e Pre and post assessment of 
knowledge and skills 

e Observation of participants 
practicing and applying newly 
acquired skills 

e Trainer performance 
feedback 


-AND- 


Training effectiveness data 

are analyzed to: 

e Inform improvements to the 
training content and delivery 


Data are not analyzed or used to 
determine effectiveness of 
training 


Training outcome 
data 


Evidence that 
data are used for 
improvements 
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effectiveness data 


DITs is assessed three times a 
year 
-AND- 


At least four sources of data are 

used: 

e = District Capacity Assessment 
(DCA) data 

e DIT observations 

e =Product reviews 

e Coaching satisfaction survey 
results (coaching recipients) 

e Coaching service delivery 
adherence data 

e Coaches’ self-reflections 
using data 

e Coaching logs 

e El Fidelity measure data 


-AND- 


Coaching effectiveness data are 
used to inform improvements in 
coaching, recruitment and 
selection, training, and other 
implementation supports 


DITs is assessed at least annually 
-AND- 


At least two sources of data are 
used from the 2-point response 


-AND- 
Coaching effectiveness data are 


only used to inform 
improvements in coaching 


DITS is not assessed and multiple 
sources of data are not used 


RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
24. RIT uses coaching | The effectiveness of RIT coaching | The effectiveness of RIT coaching | The effectiveness of RIT coaching | Coaching 


effectiveness 
data such as staff 
satisfaction 
surveys 


Evidence the 
data are used to 
inform 
improvements 
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-AND- 


The plan includes: 

e List of internal and external 
stakeholder groups (e.g., 
internal staff represented in 
the organizational chart, 
necessary outside agencies, 
families) 

e Person(s) responsible for 
communication with each 
group 

e Frequency of communication 
(e.g., following each monthly 
team meeting) 

e Type of information to 
disseminate and gather 
identified stakeholders 

e Methods of communication 
(e.g., regularly scheduled 
meeting, email) 

e Communication 
effectiveness survey and 
timelines for gathering 
communication survey data 

-AND- 

Plan is consistently used 


- AND - 
Stakeholders report that 
communication has been 
effective 


-AND- 


The plan focuses primarily on: 

e List of internal and external 
stakeholder groups (e.g., 
internal staff represented in 
the organizational chart, 
outside agencies, families) 

e Persons responsible for 
communication with each 
group 

e Frequency and methods of 
communication 


-OR- 


The plan is inconsistently used. 


-OR- 


Stakeholders are reporting 
communication to be ineffective 


RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
25. RIT uses a The plan is written and The plan is in the process of There is not a plan for Communication 
communication accessible to all regional staff being written and accessible to communication or the plan is not | plan 
plan all staff in use 


Stakeholder 
report 
summaries 
indicating 
communication 
has been 
effective 
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RCA Item 


Fully in Place 


Partially in Place 


Not in Place or Not Yet 


Potential Data 


internal barriers 


identify and address barriers 
-AND- 


The process is consistently used 
to remove internal barriers (e.g., 
policy and guidance documents 
are revised to support new ways 
of work, resources are 
allocated/re-allocated, 
competing or ineffective 
initiatives are modified or 
removed) 


-AND- 
Follow up takes place to 


determine if the way the barrier 
was addressed was effective 


-OR- 


The process is inconsistently 
used 


-OR- 


The process is not used for 
addressing internal barriers to 
prevent successful development 
of implementation capacity 


2 points 1 point 0 points Source 
26. RIT uses a process | A written process is in place The process is not written There is not a process Guidance 
for addressing outlining the steps needed to document 


outlining process 


Documentation 
showing how the 
process has been 
used (e.g., 
examples of 
identifying a 
barrier, defining 
a solution, and 
implementing 
the solution with 
effect) 
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RCA Item 


Fully in Place 


Partially in Place 


Not in Place or Not Yet 


Potential Data 
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relevant 


information to the 


State Education 
Agency (SEA) 


information from districts and 

schools to help identify: 

e Strengths of the state system 

e Barriers that need to be 
resolved at the state level to 
support regional and district 
implementation efforts (e.g., 
state/federal laws, 
mandated use of funds, 
interpretation of legislative 
language) 


-AND- 


The process is consistently used 
by REA staff and district 
executive leadership for 
reporting information to the 
identified contact liaison with 
the SEA 


-OR- 


REA staff and district executive 
leadership are unaware of how 
to use the process 


2 points 1 point 0 points Source 
27. RIT uses aprocess | A written process is in place to A written process isin place and _ | There is not a written process Guidance 
to report policy gather policy relevant is used inconsistently document 


outlining process 


Gathering of 
information from 
districts and 
regions 


Documentation 
of reporting the 
gathered 
information (e.g., 
reports, meeting 
minutes, 
agendas) 
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stakeholders a 
status report on 
capacity 
development 
efforts 


group(s) to provide a status 
report 
-AND- 


The report to identified 
stakeholder groups includes at 
least 5 of the following 7 types of 
information: 

e Number of districts across 
the region the RIT is working 
with to develop 
implementation capacity 
(e.g., DITs) 

e Each district’s stage of 
implementation 

e Capacity to develop district 
implementation structures 
(e.g., REA coaching system to 
support DITs) 

e El Fidelity data 

e Impact on student outcomes 

e Stakeholder information 
(e.g., survey data from staff) 
about implementation 
supports 

e Upcoming work to scale-up 
implementation capacity and 
continuous improvement 


-AND- 


Report is disseminated a 
minimum of twice per year 


group(s) to provide a status 
report 


-AND- 
The report includes at least 4 of 
the 7 different types of 
information outlined in the 2- 
point criteria 


-OR- 


The report is only disseminated 
once per year 


stakeholders 
-OR- 


Report focuses primarily on 
action, instead of data 


RCA Item Fully in Place Partially in Place Not in Place or Not Yet Potential Data 
2 points 1 point 0 points Source 
28. RIT provides The RIT identifies stakeholder The RIT identifies stakeholder A status report is not provided to | List of 


stakeholders 


Copy of most 
recent 
stakeholder 
status report or 
presentation 
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Action Planning 


Step 1: For any item listed below a “2” consider actions that may be completed within the next 
3 months. 


Step 2: Define the action, "who" is responsible, when it will be accomplished, and the 
team/meeting when updates on the action will be reviewed. 


Step 3: Team should prioritize the areas or items that are most critical to improve - most likely 
to improve fidelity, sustainability, and student outcomes. 


Subscale and Items Action Who When Next 
Update 


1. Leadership 


2. Action Planning 


3. Fidelity Performance 
Assessment 


4. Staff Selection 


5. Training 


6. Coaching 


7. Decision Support System 


8. Facilitative Administration 


9. Systems Intervention 


10. Stage-Based Functioning 
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Glossary 


Browse the glossary below to learn the vocabulary terms commonly encountered in the RCA. To 
successfully administer the RCA, knowledge of these terms is necessary. The glossary was compiled 
using the following resources: SISEP’s Active Implementation Hub, National Implementation Research 
Network, and PBIS.org. 


Action Planning 
A process of identifying steps or activities that must be performed well to accomplish a strategy. Action 


planning has five major elements: (1) identifying critical needs and goals; (2) identifying specific tasks 
and who will do them; (3) establishing the timeframe for when tasks will be accomplished; (4) allocating 
specific resources needed; and (5) monitoring the action items. 


Authority 
Authority in the context of the RCA refers to the power or right to make decisions regarding budgets, 
positions, and allocation of resources. 


Building Implementation Team (BIT) 
An organized and active group that supports the implementation, sustainability, and scale-up of 
Effective Innovations by integrating the use of implementation stages, drivers, and improvement cycles. 


Capacity 
Systems, activities, and resources that are necessary for schools to successfully adopt and sustain 
effective innovations. 


Coaching 
Regular, job embedded support and feedback designed to help educators use implementation science as 
intended. 


Coaching Service Delivery Plan 
A written plan detailing the frequency of coaching observations, methods of support, and routines and 
methods (e.g., written, verbal) for providing constructive feedback in a safe environment. 


Communication Plan 

A written document outlining the frequency, type, and format of communication between teams for the 
following purposes: communicate progress and celebrate success throughout the system, report 
systemic barriers that are preventing or hindering implementation and should be resolved by one of the 
groups, report on actions taken to resolve or address past issues, and revisit past decisions and 
agreements periodically to ensure that solutions are still functional. 


Competency 
Develop, improve, and sustain educators’ ability to use implementation science in order to achieve 
desired outcomes. 
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Coordinator 
Regional unit staff member assuming a lead role in preparing for and facilitating the RIT meetings, 
agenda topics, and monitoring completion of assigned actions. 


Decision Support Data System 
Processes for identifying, collecting, and analyzing data to continuously improve district implementation 
capacity. 


Diagnostic Assessment 
Assessments, which provide more in-depth information about an individual student’s specific skill, for 
the purpose of guiding future instructional supports. 


Effective Innovation 

An innovation is anything that is new to a district and that is intended for use to improve effectiveness 
or efficiency. The innovation was developed based on the best available evidence (e.g., evaluation 
results, research findings). 


Executive Management Team 

Individuals in leadership positions who are responsible for facilitating and sustaining systems change 
that includes identifying priorities, establishing policies, making decisions, providing guidance, allocating 
resources, and providing visibility for implementation efforts within the organization 


Exploration Stage 

Process to determine whether an organization is ready to engage in a change (e.g., implementation of a 
program, process, new way of doing work). A team in exploration will assess the potential match 
between the organization’s needs, the requirements of the program or practice, and the organization’s 
resources 


Facilitative Administration 
Active, intentional, and ongoing alignment of internal processes, policies, regulations, and structures to 
make new ways of work easier and more effective. 


Fidelity 
Measuring the degree to which educators use implementation science practices as intended. 


Formal 
Formal refers to an established hierarchy, procedure or set of specific behaviors. 


Full Implementation 
Selected programs, processes or new ways of doing work are considered “standard practice” because 


the new learning is now integrated into staff practices, policies and procedures. Teams in full 
implementation are routinely using the programs and processes with fidelity and by doing so, are 
experiencing good outcomes 
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Guidance Documents 
Publicly available documents outlining a process and/or procedure and its implementation. 


Implementation 

A specified set of activities designed to put into practice an activity or program of known dimensions. 
According to this definition, implementation processes are purposeful and described in sufficient details 
such that independent observers can detect the presence and strength of the “specific set of activities” 
related to implementation. 


Implementation Plan 
A detailed plan that outlines goals/objectives and strategies/actions needed for effective and sustained 
implementation and use of an El. 


Implementation Science 

Implementation science is the study of methods to promote the integration of research findings and 
evidence into policy and practice. It seeks to understand the behavior of professionals and other 
stakeholders as key variables in the sustainable uptake, adoption, implementation, and sustainability of 
Effective Innovations. 


Improvement Cycles 
Improvement cycle is a planned sequence of systematic and documented activities aimed at improving a 
process (e.g., PDSA Cycle — Plan, Do, Study, Act). 


Informal 
Informal refers to an activity or process that is marked by the absence of formality or structure. 


Initial Implementation Stage 

People are attempting to use newly learned skills related to the selected program, process or new way 
of doing work. A team in initial implementation is likely to need significantly more supports from their 
organization due to the newness of the skills they are beginning to apply to the context of their work. 
During this stage, awkwardness is associated with trying new things and the difficulties associated with 
changing old ways of work 


Installation Stage 

Resources are acquired or repurposed to accomplish the selected program, process or new way of doing 
work. A team in installation will create the infrastructure and make the changes necessary to initiate the 
new program, process or practice and use it as intended 


Interview Protocol 
A document outlining the various activities used within a selection process of a staff member. 


Leadership 
Active involvement in facilitating and sustaining systems change to support the use of Effective 


Innovations through strategic communication, decisions, guidance, resource allocation, and promotes 
vision, visibility, priority, and policy. 
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Maturity of Effective Innovations 

Innovations whose core features have been clearly defined, operationalized and have been 
implemented (past and present) with good effect resulting in staff being able to access qualified experts 
(purveyors) to provide an effective orientation, training supports, coaching and technical assistance to 
implementers that will result in successful implementation 


Operationalize 
Clearly defined behaviors an adult will do in such a way that the components are readily teachable, 
learnable, doable and assessed 


Organization 

Structures, roles, and processes for communicating clearly, analyzing data, and responding to data in 
ways that result in continuous improvement of systems and supports for educators to implement 
Effective Innovations. 


Policy Relevant Information 
Data and material that can be used to inform the development and/or refinement of a policy or 
statement of intent adopted by a Board or senior governance body. 


Practice Profile (Innovation Configuration) 

Document that identifies the core components of a program or practice and describes the key activities 
that are associated with each core component along the continuum of implementation (e.g., expected / 
proficient, developmental, unacceptable variation). Practice profiles enable a program to be teachable, 

learnable, and doable. 


Progress Monitoring 
Frequent assessment to provide more in-depth information about an individual student’s specific skills, 
for the purpose of guiding instructional supports. 


Regional Education Agency (REA) 
Public entities that provide educational support programs and services to local schools and school 
districts (Association of Educational Service Agencies, n.d.) 


Regional Implementation Team 
An organized and active group that supports the implementation, sustainability, and scale-up of 
Effective Innovations by integrating the use of implementation stages, drivers, and improvement cycles. 


Scale worthy or Scalable Practices 

Practices that have sufficient social and scientific validation to warrant the large-scale investment 
needed to transform these practices into Standard Practice. Scalable practices have documentation 
that they are needed, effective, usable, and feasible. 
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Staff Selection 

Purposeful processes for recruiting, interviewing, and hiring through an active implementation lens. This 
includes identifying skills and abilities that are prerequisites and/or specific to the use of 
implementation science, as well as characteristics that are difficult to train and coach. 


S.M.A.R.T. Goal 
S.M.A.R.T. is a mnemonic acronym, giving criteria to guide in the setting of goals and/or objectives. A 
S.M.A.R.T. goal is defined as one that is specific, measurable, achievable, relevant, and time- bound. 


Stage Based Functioning 
Process of matching intensity and types of supports and activities to districts’ stage of capacity 
development. 


Summative Assessment Data 
Measures used to gather information about student performance compared to grade level standards. 


Systems Intervention 
Intentional and ongoing efforts to influence external factors such as policies, environments, systems or 
structures that interfere with the core functions of districts and schools in the region. 


Training 
Purposeful, skill-based processes designed to support educators in acquiring the skills and information 
needed to use implementation science. 


Universal Screening 

The systematic assessment of all children within a given class, grade, school building, or school district, 
on academic and/or social-emotional indicators that the school personnel and community have agreed 
are important. 
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